Remember redline markup after CP is Applied

I want to retain (or resurrect) the redline From-To markup for any and all of the CPs that have been applied to a single requirement.

The out-of-the-box Change Proposal module does not retain the original "From" values for the attributes under CP control. So after the CP is "Applied" there is no mechanism to show the "From-To" for that particular change.

Is there a strategy that typically gets used for this application? (Is this a typical application?)

Add attributes to the CP module to store the original values?
Search the target object history for the last change to the attribute value prior to the CP create date?
Find the previous change in the CP module and use that "To" value as the next "From" ?
......  ?

Any others?

Thanks
-geoff


gBright - Thu Feb 13 01:55:26 EST 2014

Re: Remember redline markup after CP is Applied
llandale - Thu Feb 13 13:45:15 EST 2014

I suppose it would be relatively simple to add a step in the CP "Apply" procedure, to capture the From-To difference in a new CP module attribute. 

I also suppose it would be relatively simple to add a step in the CP "Create" procedure, to capture the original values of the attributes in question; but this would require a whole new set of dynamic attributes sibling to the existing "Using" and "Proposed" attributes.  In spite of the difficulties, this approach also has the significant benefit that you could adjust the "Approve" and "Apply" procedures and find requirement values that have changed since the CP was generated; thus reducing CP apply conflicts.

Searching History shouldn't be much of a problem, look for requirement changes done at the same time the CP status changed to "Applied".

In any event, it is important (I think) to distinguish between what the difference is when the CP is created, and the difference right before the CP was applied.

I had written numerous "CPS" report scripts to show what WILL happen when  the CP is applied; it did not occur to me to capture that info in attributes.

-Louie

Re: Remember redline markup after CP is Applied
gBright - Mon Feb 17 21:30:17 EST 2014

llandale - Thu Feb 13 13:45:15 EST 2014

I suppose it would be relatively simple to add a step in the CP "Apply" procedure, to capture the From-To difference in a new CP module attribute. 

I also suppose it would be relatively simple to add a step in the CP "Create" procedure, to capture the original values of the attributes in question; but this would require a whole new set of dynamic attributes sibling to the existing "Using" and "Proposed" attributes.  In spite of the difficulties, this approach also has the significant benefit that you could adjust the "Approve" and "Apply" procedures and find requirement values that have changed since the CP was generated; thus reducing CP apply conflicts.

Searching History shouldn't be much of a problem, look for requirement changes done at the same time the CP status changed to "Applied".

In any event, it is important (I think) to distinguish between what the difference is when the CP is created, and the difference right before the CP was applied.

I had written numerous "CPS" report scripts to show what WILL happen when  the CP is applied; it did not occur to me to capture that info in attributes.

-Louie

Thank you for the response, Louie.

As I think about this project I now break it down into two distinct problems:

1) How & When do you cause the original values to be captured, and
2) How/from where do you collect the original values and generate diffs.

For #2)   for the time being I've created the additional sibling attributes in the CP module with the names"From <attribute name>". Simple enough.
I actually need to concentrate more on problem #1. It's the trickier, more interesting piece.

For problem #1 (How & When to cause the original values to be captured)  my current experiments involve triggers.
Rather than sift through the DXL library to untangle the trail of function calls I opted to try attaching a pre-modify trigger to the CP attribute "CP Attr - Status." (I know nothing about the structure of the existing CPS or which dxl files to look for)

So far I have the trigger running as expected.

It's promising in that whenever the CP status changes the trigger updates the original values from the target module in the CP module "From" attributes. So the values update as the CP gets processed (Modify, In Review, Approved ...) and keeps up if the target module values get changed after CP creation.
It's also problematic in that when the CP is first created by the CPS the CP status gets updated and the trigger code runs before the CP "Create" procedure has created the link to the target requirement - so the trigger code has no link to follow to grab the original values. If I twiddle the CP status thereafter then everything is hunky dory.

Perhaps there's a more appropriate trigger event. But I don't see anything in the "Introduction to Triggers" section of Help->DXL Reference Manual that looks promising to solve the problem at initial creation. 

Another reason that the trigger approach is appealing is that sometimes we get feeling cocky and go to the dark side and (shudder) edit a CP directly in the CP module - even though it is "offically" verboten. (Some times a CP evolves and we have to alter the "proposed" values.) A trigger could fire on the proper event whether it was caused by a normal CPS procedure or someone making adjustments directly in the CP module.

Perhaps the total solution is a combination of the trigger for updates after CP creation and modifying the CP "Create" procedure to get the values at initial creation. I would certainly entertain modifying the "create" procedure - I just need the road map through the pile of DXL scripts to find it.

Thanks for your prior comments.
Any further insights would be much appreciated:
  trigger suggestions (or reasons to never do that!) and the trail to the CPS "create" procedure

Peace
-geoff

Re: Remember redline markup after CP is Applied
Mathias Mamsch - Tue Feb 18 07:39:18 EST 2014

gBright - Mon Feb 17 21:30:17 EST 2014

Thank you for the response, Louie.

As I think about this project I now break it down into two distinct problems:

1) How & When do you cause the original values to be captured, and
2) How/from where do you collect the original values and generate diffs.

For #2)   for the time being I've created the additional sibling attributes in the CP module with the names"From <attribute name>". Simple enough.
I actually need to concentrate more on problem #1. It's the trickier, more interesting piece.

For problem #1 (How & When to cause the original values to be captured)  my current experiments involve triggers.
Rather than sift through the DXL library to untangle the trail of function calls I opted to try attaching a pre-modify trigger to the CP attribute "CP Attr - Status." (I know nothing about the structure of the existing CPS or which dxl files to look for)

So far I have the trigger running as expected.

It's promising in that whenever the CP status changes the trigger updates the original values from the target module in the CP module "From" attributes. So the values update as the CP gets processed (Modify, In Review, Approved ...) and keeps up if the target module values get changed after CP creation.
It's also problematic in that when the CP is first created by the CPS the CP status gets updated and the trigger code runs before the CP "Create" procedure has created the link to the target requirement - so the trigger code has no link to follow to grab the original values. If I twiddle the CP status thereafter then everything is hunky dory.

Perhaps there's a more appropriate trigger event. But I don't see anything in the "Introduction to Triggers" section of Help->DXL Reference Manual that looks promising to solve the problem at initial creation. 

Another reason that the trigger approach is appealing is that sometimes we get feeling cocky and go to the dark side and (shudder) edit a CP directly in the CP module - even though it is "offically" verboten. (Some times a CP evolves and we have to alter the "proposed" values.) A trigger could fire on the proper event whether it was caused by a normal CPS procedure or someone making adjustments directly in the CP module.

Perhaps the total solution is a combination of the trigger for updates after CP creation and modifying the CP "Create" procedure to get the values at initial creation. I would certainly entertain modifying the "create" procedure - I just need the road map through the pile of DXL scripts to find it.

Thanks for your prior comments.
Any further insights would be much appreciated:
  trigger suggestions (or reasons to never do that!) and the trail to the CPS "create" procedure

Peace
-geoff

Hi Geoff,

the CPS files reside in lib/dxl/standard/cps as you probably know, but there are a lot of them. It should be possible to hook into the CPS by leveraging the fact, that the included routines and the GUI routines reside in different files (you can place a file between them, where you hook the CPS api functions). However that would mean, you would need to modify your deployment, i.e. this is a client specific change. Therefore unless you do not have a way to deploy this change to every computer running DOORS you can never be sure, that your code is executed on all systems. I don't know if that is acceptable for you.

The other possibility would be to try leveraging a pre-close trigger on the CPS module / requirements module that will do stuff to the module before it is saved. However I am not sure that will work reliably.

Did you ever think about grabbing the data from the history?

Just some ideas, regards, Mathias

Re: Remember redline markup after CP is Applied
llandale - Tue Feb 18 11:55:20 EST 2014

gBright - Mon Feb 17 21:30:17 EST 2014

Thank you for the response, Louie.

As I think about this project I now break it down into two distinct problems:

1) How & When do you cause the original values to be captured, and
2) How/from where do you collect the original values and generate diffs.

For #2)   for the time being I've created the additional sibling attributes in the CP module with the names"From <attribute name>". Simple enough.
I actually need to concentrate more on problem #1. It's the trickier, more interesting piece.

For problem #1 (How & When to cause the original values to be captured)  my current experiments involve triggers.
Rather than sift through the DXL library to untangle the trail of function calls I opted to try attaching a pre-modify trigger to the CP attribute "CP Attr - Status." (I know nothing about the structure of the existing CPS or which dxl files to look for)

So far I have the trigger running as expected.

It's promising in that whenever the CP status changes the trigger updates the original values from the target module in the CP module "From" attributes. So the values update as the CP gets processed (Modify, In Review, Approved ...) and keeps up if the target module values get changed after CP creation.
It's also problematic in that when the CP is first created by the CPS the CP status gets updated and the trigger code runs before the CP "Create" procedure has created the link to the target requirement - so the trigger code has no link to follow to grab the original values. If I twiddle the CP status thereafter then everything is hunky dory.

Perhaps there's a more appropriate trigger event. But I don't see anything in the "Introduction to Triggers" section of Help->DXL Reference Manual that looks promising to solve the problem at initial creation. 

Another reason that the trigger approach is appealing is that sometimes we get feeling cocky and go to the dark side and (shudder) edit a CP directly in the CP module - even though it is "offically" verboten. (Some times a CP evolves and we have to alter the "proposed" values.) A trigger could fire on the proper event whether it was caused by a normal CPS procedure or someone making adjustments directly in the CP module.

Perhaps the total solution is a combination of the trigger for updates after CP creation and modifying the CP "Create" procedure to get the values at initial creation. I would certainly entertain modifying the "create" procedure - I just need the road map through the pile of DXL scripts to find it.

Thanks for your prior comments.
Any further insights would be much appreciated:
  trigger suggestions (or reasons to never do that!) and the trail to the CPS "create" procedure

Peace
-geoff

I like this approach a lot.

I wonder if the CP Status is "New" and there is no link, your code could deploy a dynamic pre-close module trigger that copies over the requirement object's values for this CP.

I also wonder if the Status changes and there is already a "From " value, and that "From " value is not the same as the current Requirment value, that surely you would want to generate some indication that the Requirement changed AFTER the CP was generated.  Folks would definately want to know that; otherwise this CP can easily wipe out changes some other appied CP just made, which is not honky dory.

I wonder if the CP Status changes to "Applied" before or after the Requirment values are updated.

I wonder if you need some mechanism for copying over values when a "Using " attribute becomes "true"; such as may happen when the CP is reviewed.  Likewise erasing the "From " when the "Using " becomes false.

I wonder if you really only care about the "From " value when the CP is about to be Applied (or Rejected).  Showing predicted changes for "active" CPs can be done against the current Requirement Value.

-Louie

Re: Remember redline markup after CP is Applied
gBright - Tue Feb 18 18:28:38 EST 2014

llandale - Tue Feb 18 11:55:20 EST 2014

I like this approach a lot.

I wonder if the CP Status is "New" and there is no link, your code could deploy a dynamic pre-close module trigger that copies over the requirement object's values for this CP.

I also wonder if the Status changes and there is already a "From " value, and that "From " value is not the same as the current Requirment value, that surely you would want to generate some indication that the Requirement changed AFTER the CP was generated.  Folks would definately want to know that; otherwise this CP can easily wipe out changes some other appied CP just made, which is not honky dory.

I wonder if the CP Status changes to "Applied" before or after the Requirment values are updated.

I wonder if you need some mechanism for copying over values when a "Using " attribute becomes "true"; such as may happen when the CP is reviewed.  Likewise erasing the "From " when the "Using " becomes false.

I wonder if you really only care about the "From " value when the CP is about to be Applied (or Rejected).  Showing predicted changes for "active" CPs can be done against the current Requirement Value.

-Louie

Thank you for your response above, Mathias.
Yes, I was aware of the prospect of sifting through history to find the value changes just prior to a CP creation. That also would entail having to skip bakwards over Baselines if the last change occured that long ago.

For the moment let's follow Louie's train of thought about identifying and managing conflicts of multiple CPs and errant changes in the target module ...

Let's try this list of steps/functionality and see if it gives a user the desired interaction and useability:

1. Submit CP
(No original values captured while the originating session is open)
(From-To markup is generated based on values in target module using Layout DXL)
(From-To markup is immediately visible after CP submit - makes me happy)

2. Trigger on appropriate event : Module Pre-Save or Module-Open or ...  other
(Trigger scours the CP module & checks to see if values have been captured once -
  Captures original values ONLY if they have NOT been captured yet)

3. DXL Attribute tests equality of CP stored original values and the target module values
(Flag CPs where they are different)
(Does not flag CP if no original values are captured yet)

4. User/CP Manager can filter on the problematic CPs labelled above
Examines the CPs and target objects and resolves issues
  - Fix, reject, submit new CPs
  - Identify unauthorized changes to the target module

5. A user invoked function is available to recapture target module values
for a selected CP at the users' discretion.

6. A CP goes through the usual review cycle
(New, In Review, Approved, Applied) unless the CP original values and the target values deviate and are detected.

7. Recapture of target module values is NEVER allowed after the CP status becomes "Applied"
Must be coded into the trigger and user recapture functions.
The From-To state of the CP is preserved in perpetuity.
Views in the target and CP modules must be designed to selectively show markup from the proper source values (target module or CP stored values) based on the CP status - markup based on CP stored values if status = "Applied"

How does that feel?

Seems to offer a fairly smooth user interaction with some useful features.
No client deployment changes.
And all of the coding techniques are low hanging fruit for an intermediate level DXL slinger like me.

I smell a state transition diagram in the offing.

Thoughts?

peace
-geoff

Re: Remember redline markup after CP is Applied
llandale - Thu Feb 20 15:14:30 EST 2014

gBright - Tue Feb 18 18:28:38 EST 2014

Thank you for your response above, Mathias.
Yes, I was aware of the prospect of sifting through history to find the value changes just prior to a CP creation. That also would entail having to skip bakwards over Baselines if the last change occured that long ago.

For the moment let's follow Louie's train of thought about identifying and managing conflicts of multiple CPs and errant changes in the target module ...

Let's try this list of steps/functionality and see if it gives a user the desired interaction and useability:

1. Submit CP
(No original values captured while the originating session is open)
(From-To markup is generated based on values in target module using Layout DXL)
(From-To markup is immediately visible after CP submit - makes me happy)

2. Trigger on appropriate event : Module Pre-Save or Module-Open or ...  other
(Trigger scours the CP module & checks to see if values have been captured once -
  Captures original values ONLY if they have NOT been captured yet)

3. DXL Attribute tests equality of CP stored original values and the target module values
(Flag CPs where they are different)
(Does not flag CP if no original values are captured yet)

4. User/CP Manager can filter on the problematic CPs labelled above
Examines the CPs and target objects and resolves issues
  - Fix, reject, submit new CPs
  - Identify unauthorized changes to the target module

5. A user invoked function is available to recapture target module values
for a selected CP at the users' discretion.

6. A CP goes through the usual review cycle
(New, In Review, Approved, Applied) unless the CP original values and the target values deviate and are detected.

7. Recapture of target module values is NEVER allowed after the CP status becomes "Applied"
Must be coded into the trigger and user recapture functions.
The From-To state of the CP is preserved in perpetuity.
Views in the target and CP modules must be designed to selectively show markup from the proper source values (target module or CP stored values) based on the CP status - markup based on CP stored values if status = "Applied"

How does that feel?

Seems to offer a fairly smooth user interaction with some useful features.
No client deployment changes.
And all of the coding techniques are low hanging fruit for an intermediate level DXL slinger like me.

I smell a state transition diagram in the offing.

Thoughts?

peace
-geoff

Does not look like "low hanging fruit" to me.  But yes you won't need a fork-lift either.

1.  Intense code is better for Attr-DXL than layouts.
2.  Trigger event is tricky, there is no such "module-save" event.  ("Save" is an attribute-value thing only). 

If it were me, I'd have some nightly on-demand batch program making these updates.  I'd also perhaps play with the notion of violating some sacred law, and possibly have your clever attr-DXL have the ability to set un-set "Original" values; attempting to do so only if it currently "canModify" the CP Object.  Be advised you will need to be able to handle an original null value as being "valid"; not sure how best to do that.

4.  5.  I'm feeling these should be merged since they are all part of a particular CP reviewer's chore.  It will require some really smart subjective guidelines on how to resolve these potential sabotages.  One way is to update the "Proposed" value to include the recent change made to the requirement.  Another way is to back-out the requirement change, and revoke the DOORS rights of the clown who edited it.

5.  Something doesn't "feel" quite right about updating the CP's "Original" value with the approved requirement's "Current" value.  I cannot help but think that if we want to save the "Original" that we indeed want to preserve the "Original" "Original" value, as well as the value against which the CP is currently "blessed" against.  But this is probably just me.

5b.  thinking out loud: add "CP Verified By" and "CP Verified On" attributes to the CP module, where your knowlegeable reviewer can indicate the CP is conflict free as of a certain date.  Mmmmmmm.

7.  ... or "Rejected".

-Louie

I have actually implemented the notion of looking for "Sabotages" of CPs, by comparing the Last-Modified-Date of an object-attribute (e.g. the last time "Object Text" changed for object Sys_101); the "Current" requirement value with the corresponding "Proposed" value.  Folks "approve" a recent Requirement Change by refreshing the corresponding "Proposed", giving it a more recent date than the Requirement attribute.  I also implemented looking for "Clash", where two active CPs propose different changes to the same object-attribute value; that one just looks at all the active CPs.  Did a 3rd check reporting Requirements that have an active "Delete" and also some other CP.  Called that a "Collision", for which I rightly recieved a lot of "Derision".

Re: Remember redline markup after CP is Applied
smarti.sj - Fri Feb 21 08:06:10 EST 2014

Very interesting posts - thank you - we set access controls at the Author level in the CP modules which keeps the chaos down (authors only have write access to their own CPs).  We also allow the original authors to update CPs/errors after they are written.  Once CPs have been "approved" we run a script to move them to an area that is locked down and that is when we also deal with fixing the "usings" to reflect possible updates.  - I would suggest that is also possibly the time to run a "capture changes" script.

Bottom line is the CP System is in dire need of an update that more closely mimics a change environment (complete with access controls, redlines etc) as opposed to isolated change mechanics.

Re: Remember redline markup after CP is Applied
gBright - Fri Feb 21 13:45:43 EST 2014

Great thoughts everyone.

Incorporating the latest thoughts  ...

1. Submit CP
No new functionality. No original values captured while the originating session is open

2. Trigger on an appropriate reliable event to capture "From" values: (Module Open,Status change,  nightly batch ...)
Snag the original values at the "best" point in the workflow without intruding on users to do somehting specifc (it should "just happen")
Trigger scours the CP module & checks to see if values have been captured once -
Captures original values ONLY if they have NOT been captured yet

3. DXL Attribute tests equality of CP stored original values and the target module values
Other functionality can be developed (scritps or DXL attributes) to expand error checking for things like Louie's Clash and Collision.

4. User/CP Manager can filter on the problematic CPs labelled above
Examines the CPs and target objects and resolves issues

5. (DELETE THIS) A user invoked function is available to recapture target module values

6. A CP goes through the usual review cycle
(New, In Review, Approved, Applied) unless the CP original values and the target values deviate and are detected.

7. Recapture of target module values is NEVER allowed after they are captured the first time
Must be coded into the trigger function.
But, maybe it's a case of "Never say never." Might want to add that "recapture" function from #5 as soon as I stub my toe on a circumstance I'm not aware of yet. But I'll wait till then.

That feels like a pretty good useable flow for my situation. It seems to find the problems and preserves the "from-to" of a change after newer CPs are applied later to the same requirement. There's just that narrow window that could allow an illegitimate change to the target module after CP creation but before the trigger runs to capture "From" values. So be it.

In order to get this far I only need to think about 3 specific chunks of DXL functionality:

  • Markup views: Red/Blue diffs are generated from the live target values in the target module until the "From" (original) values are captured in the CP.
  • DXL attribute(s) for error checking in the CP - flag CPs where "From" (original) values in the CP differ from the live arget values until the CP is Applied.
  • Trigger or automated batch DXL to snag the "From" values at the earliest practical time without intruding on users (don't need to remember to run a specific function or have the CP module open in edit mode ... etc)
    Lots of options for experimentation here - finding the best point in the work flow to actually capture "From: values.
    Could be ASAP (next module "Open") or when CP Status changes to "Approved" ... the possibilities are endless.
    Code should be the same - just different trigger points.

This is an "architecture" where I can get started. Manageable with some flexibility.

Thanks for the ideas and validation.

If you have any comments or ideas on any details of those 3 functionality areas please feel free to reply. I'll gladly check 'em out.

Peace
-geoff

Re: Remember redline markup after CP is Applied
llandale - Fri Feb 21 14:09:56 EST 2014

gBright - Fri Feb 21 13:45:43 EST 2014

Great thoughts everyone.

Incorporating the latest thoughts  ...

1. Submit CP
No new functionality. No original values captured while the originating session is open

2. Trigger on an appropriate reliable event to capture "From" values: (Module Open,Status change,  nightly batch ...)
Snag the original values at the "best" point in the workflow without intruding on users to do somehting specifc (it should "just happen")
Trigger scours the CP module & checks to see if values have been captured once -
Captures original values ONLY if they have NOT been captured yet

3. DXL Attribute tests equality of CP stored original values and the target module values
Other functionality can be developed (scritps or DXL attributes) to expand error checking for things like Louie's Clash and Collision.

4. User/CP Manager can filter on the problematic CPs labelled above
Examines the CPs and target objects and resolves issues

5. (DELETE THIS) A user invoked function is available to recapture target module values

6. A CP goes through the usual review cycle
(New, In Review, Approved, Applied) unless the CP original values and the target values deviate and are detected.

7. Recapture of target module values is NEVER allowed after they are captured the first time
Must be coded into the trigger function.
But, maybe it's a case of "Never say never." Might want to add that "recapture" function from #5 as soon as I stub my toe on a circumstance I'm not aware of yet. But I'll wait till then.

That feels like a pretty good useable flow for my situation. It seems to find the problems and preserves the "from-to" of a change after newer CPs are applied later to the same requirement. There's just that narrow window that could allow an illegitimate change to the target module after CP creation but before the trigger runs to capture "From" values. So be it.

In order to get this far I only need to think about 3 specific chunks of DXL functionality:

  • Markup views: Red/Blue diffs are generated from the live target values in the target module until the "From" (original) values are captured in the CP.
  • DXL attribute(s) for error checking in the CP - flag CPs where "From" (original) values in the CP differ from the live arget values until the CP is Applied.
  • Trigger or automated batch DXL to snag the "From" values at the earliest practical time without intruding on users (don't need to remember to run a specific function or have the CP module open in edit mode ... etc)
    Lots of options for experimentation here - finding the best point in the work flow to actually capture "From: values.
    Could be ASAP (next module "Open") or when CP Status changes to "Approved" ... the possibilities are endless.
    Code should be the same - just different trigger points.

This is an "architecture" where I can get started. Manageable with some flexibility.

Thanks for the ideas and validation.

If you have any comments or ideas on any details of those 3 functionality areas please feel free to reply. I'll gladly check 'em out.

Peace
-geoff

It's Friday and it's time for Mindless Rambling:

  • Seems like we could get some simulation for an "Object Create" trigger in the CP modules.  If there is no such Reminder Dialog showing then it "show"s a Reminder Dialog but does not otherwise interfere with what is going on (CP create).  The Reminder Dialog displays a "Refresh my Original Values" button.  the User, after creating the CP, could push the button.  This dialog is tied to the open module, closes when the CP module closes; and the existance of the Dyanmic Trigger that does that could be the check for whether such a dialog is open.
  • On the other hand, the script (whenever it runs) that initializes the Original Value can be written to scan History and look at Last Modified On values; it would thus get the correct Original value even the requirement change after CP but before the script runs.
  • I don't think a pre-close-module Trigger for the CP module would work as there would be conflict between closing a saved CP module, and closing one without saving changes.
  • I think there needs to be an interim "Recent Requirement Change is OK" ability.  The Requirement Changes, the reviewer takes a look and approves it.  That "Approval" would take the form of refreshing the "Original" text with the current one; effectively erasing the difference.  So I'm thinking perhaps you need your step 5.
  • I wonder if Text and Heading are the only attributes you really care about capturing.
  • You will need some ability to make sure these extra attributes exist in the current CP module.  And, I suppose if they de-configure an attribute; to delete such attributes.
  • Oops, don't allow changes when CP is Applied, Rejected, or Approved.

I'm rather impressed with this entire effort.  It is pretty rare for folks to think all this stuff through before they start coding.  Including me, BTW ;)

-Louie